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ABSTRACT 

A model for evaluating a college second language 
writing instruction program is presented, based on experiences at the 
De La Salle University (Philippines) writing laboratory. The 
laboratory, a support unit of the university's language program, 
provides remedial instruction in composition and academic writing for 
referred students. The proposed evaluation model is longitudinal in 
design, and focuses on three program effects: (1) student cognitive 
gains, as measured by improvement in writing on essays; (2) student 
attitudes toward writing and toward the program itself, expectations 
of instruction, continuing motivation, and usefulness of materials 
and strategies learned; and (3) instructional delivery systems, 
through observation of teaching and monitoring of student portfolios. 
A pilot application of the model at the laboratory is described, 
although it is noted that its sampling is inadequate for use as an 
actual assessment. Appended materials include a writing pretest, 
three attitudinal scales, and the final (posttest) composition 
exercise. (MSE) 



ft ft ft ft ft ft ft ft * ft * Vc Vc ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 

ft ft ft ft Vc ft ft ft Vc ft Vc ft ft ft ft ft ft ft ft ft ft Vc ft ft Vc ft ft ft ft ft ft ft ft ft ft ft ft ft ft Vcft ft ft ft Vc ft ft ft ft ft ft ft ft ft Vc ft ft Vc ft ft Vc Vc ft Vc ft ft Vc ft ft ft Vc 



TOWARDS EVALUATING THE WRITING LABORATORY: 

A PROTOTYPE 



Ma. FlorEL Mejorada, 
Elvira Fonader 



INTRODUCTION 



Writing has been a major concern of researchers, language teachers and materials writers. 
A body of research has been done on the teaching of writing as a process (Flower 1980; 
Jacobs 1982; Zamel 1982, 1983; Raimes 1983, 1985;) which led to formulating appropriate 
strategies of responding to students' compositions; and eventually setting up scoring or rating 
techniques (Heaton 1975; Perkins 1983; Jacobs 1981 in Hartfiel et al. 1985; Josephson 1989). 
However, no one had so far come up with an evaluation scheme that includes various 
components of a writing program with which students are taught. 

This paper proposes a model which attempts at a composite evaluation of the writing 
program. As such, it includes the learner's attitudes towards writing and the program, the 
progress he attains at the lab, and the delivery system of the program. 

The proposal focuses on evaluating the Writing Laboratory (WL) of De La Salle 
University (DLSU) in Manila, Philippines. 
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A BRIEF DESCRIPTION OF THE PROGRAM 

The Writing Laboratory of DLSU is a support unit of the Languages Department that 
accommodates students with writing problems. These students are recommended by their 
teachers for remediation. They take remedial classes for fifteen hours in the lab, spread over 
five weeks, meeting twice a week. They are taught by teachers who use the one-to-one 
conference-centered strategy. As there are only two to three students handled at one session 
the set-up makes immediate feedbacking possible. The WL's aims are to develop effective 
composing strategies and patterns of writing behaviour, to develop proofreading and editing 
skills and to facilitate the satisfactory writing and revision of academic papers. 
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SIGNIFICANCE OF THE MODEL 

While the model reflects our perceptions of ihe Writing Laboratory, we hope to 
provide a framework which can be applicable to other writing contexts beyond the WL of 
DLSU. 

The model speculates that the evaluation is primarily intended to help the program 
staff composed of teachers who actually conduct the lab and the program coordinator to 
examine the effectiveness of the program and of its identifiable subcomponents. The results 
will necessarily make them rethink assumptions that underlie the activities conducted in the 
lab. Secondarily, the evaluation will inform the administrators of the status of the program 
so that they can lend support in terms of deciding alternative courses of action to make the 
program serve best its target clientele. 



SCOPE AND LIMITATION OF THE MODEL 

Viewed from the perspective of an on-going program delivery, the model covers three 
major components: affective evaluation, cognitive, and evaluation of delivery systems 
(Hcnning 1987). It is deemed that knowledge of the indexes of these t! ■• components 
could indicate the impact of the program on its users who are the students and its 
implementors who are the teachers as well as administrators. 
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To facilitate the modelling scheme, this prototype takes only twenty students whose 
attitudes toward the program and whatever skills they shall have attained will be presented. 
However, the model will not go into statistical details of the results of iie questionnaires and 
tests given to them. Since the sampling was not representative of all the classes from where 
the lab population came, the model is not going to present an actual evaluation of the lab's 
achievement. Besides, cutting up the limited sample and analyzing against various 
components would only lead to finding something "significant" just by chance, and the small 
numbers would likely yield unstable interpretations. Therefore, considering these 
limitations, the model will only give a general view of the program's impact on the users. 



METHODOLOGY 

Responde nts of the study . Through a pretest/diagnostic test, and a week's 
observation of class performance the teacher determines the students with writing problems 
and recommends them to the lab. However, enrollment at the lab is voluntary. So the 
students who actually register are presumed to have some degree of motivation which those 
of similar category do not have. 
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The Design. Figure 1 illustrates the evaluation scheme which is both longitudinal and 
cross-sectional. It is longitudinal because it assesses the major components from the 
beginning through the last day in the lab; it is cross-sectional because it examines these 
components after the fifth meeting, that is the mid -part of the program. Here, the student's 
individual progress is considered, not for purposes of computing his grades but for 
correlating his written products with his realizations of the lab after getting acquainted with 
the materials and the strategics with which his/her instructor carries out activities. 
Additionally, these current discoveries will be interpreted in relation to records that arc 
available in the portfolio. 

Affective evaluation is carried out by means of a Likert-typc questionnaire 
administered to students three times, the first one is to elicit feedback about their attitudes 
toward writing and to the program, and their expectations in the lab. The second and the 
last questionnaires arc to know about their continuing motivation, which materials they find 
useful and which strategics they think work out for them. While the program may effect 
some attitudinal changes, the evaluation does not aim at statistically measuring them. It 
assumes that any positive attitudinal changes the learners have will be generally reflected in 
their writing performance. Besides, it is not customary to expect radical attitude gains in a 
short period. 

Cognitive evaluation is determined through a pre/post test comparison. On the first 
meeting the student is shown the paper he wrote in class. His strong points and his 
weaknesses revealed in his composition are analyzed. He is then assured that his weaknesses 
can be remediated if he works hard together with his/her lab instructor. On the last day, he 
is asked to do a post test designed to be of equal difficulty to the pretest and the same test is 
also taken by those recommended by the teacher to the lab but chose not to enroll. 
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The posttest scores of the program takers also serve as an index to whether the 
instructional objectives were met or not. Questions on the appropriacy, measurability and 
attainability of the aims of the Writing Lab will be proved or disproved by the same results. 

Evaluation of delivery systems is done through a portfolio monitoring done by the 
teacher, teacher ratings by students, and observations by fellow teachers and the WL 
Director, and a bi-weekly subject teacher-lab instructor conference. The teacher writes 
notes or keeps anecdotal records on each of her students as the instruction progresses. 
These notes are available to other teacher* for purposes of comparing experiences and 
sharing information that are of common concern. The students' assessment is taken from 
their answers to question on strategies and materials in the attitudinal questionnaire. 
Although teacher ratings by students "have offered problems in that they have been found to 
be more indicative of students' needs or desires than they are of teaching quality" (Henning 
1987: 150), they are considered valuable in the lab to elicit reactions to the particular 
strategies not used in the regular classroom. Teacher observation done by the WL director 
is also of great value. The post observation conference is beneficial to both the teacher and 
the observer in terms of recognizing the program's strengths and scrutinizing problematic 
areas that call for new directions. Peer observation enables the instructor and fellow 
instructors to share with one another their on-going experiences with their respective 
students particularly on techniques and materials that have either proved successful or 
otherwise on certain writing problems of students. 

Forty students with below 70% had similar ratings in the different components of the 
composition test they took on their first day in the regular English course. 

Table 1: Pre Test Results of Students with Scores below 60X 
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AFFECTIVE EVALUATION 

This evaluation considers the lab takers only. Since the focus is on what the program 
offers, the main aim for giving the questionnaire is lo elicit feedback in order to give the 
maximum help to the students who have opted to be in the program. 
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IMPLEMENTING THE MODEL 



RECOMMENDING STUDENTS TO THE LABORATORY 

The regular classroom teachers, identified the students who needed the program. 
This was facilitated through the diagnostic test which they checked together with a lab 
instructor as interater. (Please see Appendix A, page 9, for a copy of the test). Moreover, 
the teachers' week-long observation of student performance in class would make them surer 
of who to recommend to the lab. 

The scoring of the composition was based on Jacobs' ESL Composition Profile which 
includes the following components: Content (30%), Organization (20%), Vocabulary 
(20%), Language Use (25%) and Mechanics (5%). Because DLSU's passing cut-off score in 
the academic courses is 70%, those who got 69% and below on the diagnostic test were 
considered "failures" and therefore, needed the program. 

Of the forty students recommended, twenty students actually took the course. They 
composed the experimental group (Ex) for the model. These students came from two 
sections of English for Computer Science (EnComp), section of English for Liberal Arts 
(EnglArt), and two sections of English for Commerce (ComArt2). The EnComp and 
EnglArt courses were under one teacher while the ComArt2 courses were handled by 
another. All of these English courses were basically writing with research along respective 
fields as focus. The rest of the poor writers, numbering twenty from the same classes, for 
whom the program should have been suitable opted not to take the program. These twenty 
students served as the control group (Cg). Table 1, p. 3 shows that. 

On the first meeting with the lab instructor, the students answered a questionnaire 
(See Appendix B, .10). The answers indicated positive attitudes towards the program with 
majority claiming that the objectives of the lab were made known to them and that their 
coming to enroll was not because of any teacher pressure or classroom requirement. After 
the fifth meeting, these students had to answer another set of questions. This time the 
questionnaire was much longer since the purpose was to gain feedback about how the 
program was meeting the students' needs. (See Appendix C, pp. 12-13). On the last 
meeting, a questionnaire (Appendix D, p. 15) was answered again for final attitudinal 
assessment. Moreover, the results were expected to hint to the WL staff about what to keep, 
what to disregard and what to add to the program. 

Table 2 below summarizes the results of the pre-laboratory, mid-laboratory and post- 
laboratory exposures survey of attitudes and continuing motivation of students. The means 
for the three surveys recorded values approximating 4.0 on the scale which could be 
interpreted as learners having positive attitude towards the program and having a rather 
stable degree of continuing motivation throughout the five-week lab period. 

Table 2: Mean Scores on Attitude Survey 

j • i i 

Score! Pre-Lab \ Mid- Lab! Post-Lab \ Mean Ave. 

jiii 
Mean I 3.977 \ 3.866 ! 4.014 ! 3.952 



COGNITIVE EVALUATION 

This component assessed the entry-level and exit-level performance of students 
(Please see Appendix E, p. 17 for the post test). Table 3 presents the post test results of WL 
students. 




Table 3: Posttest Results of Students in the Writing Lab 
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Figure 2, p. 6, presents a more detailed picture of students' writing skills noting their 
improvement in various components of the composition. Content (3.5 gain score) and 
Language use (3.7) recorded the first two highest gain scores. On the other hand, the scores 
of the Control group on the same components were much lower. Content recorded a 1.08 
gain score and Language Use, 1.1. 

Comparing the overall scores of the two groups, the lab takers registered a higher 
degree of improvement over those who did not take WL lessons. Figure 3, p.7 presents the 
picture of both the experimental and the control group on their pre/post performance with 
their respective gain scores. With the mean pretest scores obviously registering no 
significant difference between the two groups, the experimental group's posttest scores, 
which are higher than those of the non-lab takers (76.82 overall posttest score with a gain of 
10.55 for the Ex group, and 69.58, with a gain of 4.23 for the C group), may permit the 
conclusion that their gain score can be attributed to their participation in the program. Of 
course, this does not discount the possibility that there might have been initial differences 
between the two groups which could not be identified, and that these differences could have 
accounted for some of the posttest differences. 
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APPENDIX A 



Pre-Tcst 



Direction: Study the picture below, and then write an essay of at least three paragraphs 
developing ; the topic, The Fate of the Filipino Scientists Today- for the feature 
section of Bulletin Today (Daily newspaper). 
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APPENDIX B 

Writing Laboratory Attitudinal Scale A 

Instructions: Answer the following questions by shading the circle that corresponds to your 
answer in the sheet. The answer sheet has the following scale: 5 (Strongly Agree); 4 
(Agree); 3 (Neutral); 2 (Disagree); and 1 (Strongly Disagree). 

1 I expect to develop my writing skills in the laboratory. 

2 The objectives of the program have been made clear to me. 

3 I expect to identify my writing weaknesses. 

4 I will attend the lab session regularly. 

5 I am open to criticism regarding my writing. 

6 I will make use of the lab resources to my advantage. 

7 My going to the lab is of my own initiative. 

8 I attend the lab to fulfill a requirement. 

9 I enrolled in the lab because of teacher-pressure. 

10 I do not feel comfortable with my writing. 
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APPENDIX B (Cont'd) 



Teacher : Section 

Forn A 

5 4 3 2 1 

1. 0 0 0 0 0 

2. 0 0 0 0 0 

3. 0 0 0 0 0 

4. 0 0 0 0 0 

5. 0 0 0 0 0 

6. 0 0 0 0 0 
7-0 0 0 0 0 

8. 0 0 0 0 0 

9. 0 0 0 0 0 

10. 0 0 0 0 0 
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Writing Laboratory Attitudinal Scale B 

Instructions: Answer the following questions by shading the circle that corresponds to your 
answer in the answer sheet. The answer sheet has the following scale: 5 (Strongly Agree); 4 
(Agree); 3 (Neutral); 2 (Disagree); and 1 (Strongly Disagree). 

1 The materials are helpful in attaining the objectives of the program. 

2 The instructor helps me improve my writing abilities. 

3 I find the program addressing my writing weaknesses, 

4 I feel that my writing skills have improved in the lab. 

5 The time I spend in the lab is worthwhile. 

6 I find the length of time I spend in the lab sufficient. 

7 The one-on-one instruction is helpful in my achieving the objectives of the course, 

8 The quality of materials meets my expectation. 

9 The instructor explains well and to the point. 

10 My writing has improved since I attended the lab. 

11 I am getting my money's worth in the lab. 

12 My writing had developed significantly with the program. 

13 The self-access materials were sufficient to address my needs. 

14 I have developed confidence in my writing abilities. 

15 I have learned to evaluate my own writing and revise it accordingly, 

16 I find the instruction in the lab adequate. 

17 I find my going in the lab a waste of time. 

18 There are materials which 1 find difficult or irrelevant. 

19 The individualized instruction is not helping me at all. 

20 The 15-hour session in the lab is too short to enable me to develop my writing skills. 

21 1 feel that the one-and-a-half hour sessions should be longer, 

22 My instructor is most of the time unprepared for the session. 

23 Choosing materials in the self-access collection is difficult. 

24 The lab needs to develop belter materials for instruction. 

25 There is a mismatch between my needs and what the lab has to offer. 

26 The objectives of the lab were not clear to me. 

12 
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27 I am not able to develop my writing skills despite the program offerings. 

28 My instructor does not provide enough feedback to allow me to see my progress. 

29 The lab program needs further development. 

30 I fell that I am getting poor quality instruction in the lab. 

31 My writing has remained the same as when I started in the lab. 

32 I don't feel that the lab is able to address my writing problems. 

33 I always feel threatened everytime my instructor comments on my composition. 

34 My morale has been very low since I attended the lab sessions. 

35 I feel at ease working in the lab. 

36 I feel at ease conferring with my instructor in the lab. 

37 I am bored with the way my instructor conducts the sessions most of the time. 

38 The lessons are repetitive of those I have in my English course. 

39 The lessons in the lab complement those In my English course. 
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APPENDIX C (Cont f d) 



Teacher: Section 

Form B 
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APPENDIX D 



Writing Laboratory Attitudinal Scale C 

Instructions: Answer the following questions by shading the circle that corresponds to your 
answer in the answer sheet. The answer sheet has the following scale: 5 (Strongly Agree); 4 
(Agree); 3 (Neutral); 2 (Disagree); and 1 (Strongly Disagree). 

1 With the program, I can now write a well-organized composition. 

2 I feel that I wasted my money in the lab. 

3 The materials to work on were those that I did not need. 

4 The lab program is an unnecessary support of the E nglis h course. 

5 I am confident of what I am doing in the lab most of the time. 

6 I feel that I am not using my time well in the lab. 

7 Until now, I could not write a composition that meets the standard of my instructor. 

8 My instructor has been very supportive throughout the program. 

9 I feel I have accomplished very little in the lab. 

10 The lab sessions have been productive. 
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Teacher : 
Korm C 
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APPENDIX E 



Final Composition 



Direction: A local newspaper is inviting you to write an editorial for its next issue. Based 
on the picture below, write an essay of at least three paragraphs answering the 
question, "What Makes an Employee Unproductive?" 
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